Chapter 12 Safety
	First, do no harm
	Vehicles, nuclear power plants, medical
	May directly control systems that must be safe
	May monitor systems that must be safe

NEVER certain system is free of faults
	(non-computer systems have issues too)
	Want to reduce safety risks as much as possible
Many errors come from incomplete or misunderstood specifications
Many safety problems come from more than one problem happening at once
Hardware malfunctions

12.1 safety critical systems
	Primary--System can directly cause harm
	Secondary—indirect harm 
(poor system design process can lead to unsafe system)

Hazard avoidance
Hazard detection and removal
Damage limitation

Critical systems operate safely almost all of the time

12.2 safety requirements
	Software may not take into account a rare situation
	Safety is NOT about normal operation
	Shut down is often safest
Hazard identification
	Physical, software, electrical….
Hazard assessment
	What could cause? 
	(acceptability)
		Intolerable
		ALARP as low as reasonably possible
		Acceptable
Hazard analysis
	Fault tree analysis
		Work backward from hazard
		How did we get there?
		For each way to reach the hazard, 
find what would prevent it from happening
risk reduction
	hazard avoidance
	hazard detection and removal
	damage limitation

12.3 safety engineering processes
	Additional verification
	Safety reviews
	Safety assurance
		Check that system operates safely
		Name individuals who are in charge
		Formal verification (mathematical )
		Model checking (FSA finite state automata)
		Static analysis
		User defined error checking
		Assertion checking

12.4 safety cases
	Include safety in design process
	Structured arguments
	How can system be vulnerable?
	How did we get there?
	How can we stop from going there?